Federated Learning with Lossy Distributed Source Coding: Analysis and Optimization

Recently, federated learning (FL), which replaces data sharing with model sharing, has emerged as an efficient and privacy-friendly machine (ML) paradigm. One of the main challenges in FL is huge communication cost for aggregation. Many compression/quantization schemes have been proposed to reduce However, following question remains unanswered: What fundamental trade-off between convergence performance? In this paper, we manage answer question. Specifically, first put forth a general framework aggregation performance analysis based on rate-distortion theory. Under framework, derive inner bound region We then conduct connect distortion performance. formulate minimization problem improve Two algorithms are developed solve above problem. Numerical results distortion, performance, demonstrate that baseline still great potential further improvement.